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(54) Method and device for filtering a picture signal, and encoding/decoding using the same 



(57) A great decrease of a step, i.e., block distortion, 
of such as antifact brightness and color, on a block 
boundary, leads to much visual improvement of picture 
quality. 

Against a picture that has been encoded and 
decoded by block, the smoothing filter characteristics is 
changed according to quantizing parameter, block activ- 
ity of a decoded picture, the activity of each pixel of the 
decoded picture, and the position with respect to a block 
boundary of a pixel. A smoothing filter utilizes a 
weighted mean whose weight is the inverse number of a 
difference between a target pixel and its surrounding 
pixels. The weighted mean is obtained by adding a cer- 
tain number to a weight before taking its inverse 
number. The value to be added is controlled by the 
parameter. Similarly, the mixing ratio of a pixel value 
after performing the smoothing filter and a decoded 
pixel value is changed depending on the parameter. A 
filter having strong edge preserving property is utilized 
for the inside of a block, and a filter having weak edge 
preserving property is utilized for a block boundary. A fil- 
ter having edge preserving property is utilized for the 
post/loop filter. 
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Description 

FILED OF THE INVENTION 

5 The present invention relates to a picture signal smoothing device, i.e., a picture filter, and its method in which 

against a picture that has been encoded and decoded by block, a smoothing filter processing whose strength changes 
depending on quantizing parameter and block activity of a decoded picture, is performed on each pixel position to 
obtain high quality picture free from antifact noise; and a picture encoding device and its method, and a picture decod- 
ing device and its method, all of which utilize the above filter as a postfilter or a loop filter. 

10 

BACKGROUND OF THE INVENTION 

In order to encode a picture signal at high efficiency, there has been frequently used a moving picture encoding and 
decoding system and its device that appropriately encode a picture signal mainly by block. As a typical one, TV Confer- 

75 ence system based on International Telegraph and Telephone Standard (ITU-T H. 261 standard) is generally used. By 
such encoding systems, however, a block-shaped step, i.e., block noise, due to quantizing error is perceived in a 
decoded picture. Further, it is known that if there is a rapid pixel change such as a character on a flat background, 
pseudo edge called as mosquito noise occurs in its surrounding. The block-shaped noise can be observed when a step 
of antifact brightness, color and the like occurs on a block boundary. In prior art techniques, a smoothing filter for 

20 decreasing such a step is utilized against the entire decoded picture or a pixel exists on the block boundary, to reduce 
the block-shaped noise. 

Figure 24 is a block diagram of an inter-frame encoding and decoding system of the ITU-T H. 261 standard, and 
indicates the position at which block-shaped noise reducing process is performed. In this figure, a subtracter 1 1 gener- 
ates a difference between an input picture signal 10 and a picture signal 24 of the previous frame that has been 

25 encoded, decoded and reconstituted. The difference is encoded in an encoder 12 by means of DCT. quantization and 
Huffman coding. An encoded error signal 13 is transmitted to a decoder 25, and is again returned to a difference signal 
in a local decorder 14 by means of inverse Huffman coding, inverse quantization and inverse DCT. This difference sig- 
nal 15 contains operation errors in the encoder 12 and the local decorder 14, resulting in a value different from the dif- 
ference generated in the subtracter 1 1 . 

30 Then, the difference signal 15 and the picture signal 24 of the previous frame are added in an adder 16 to generate 
a picture signal 17, which is then stored in a frame memory 18, standing by for the next picture signal input. In these 
operations, the processing is performed by block of a picture divided into 8x8 dot size. 

When taking, in the subtracter 1 1 , the difference between the input picture signal 1 0 and the previous picture signal 
24 from the frame memory 1 8, there is generally employed technique called motion estimation and compensation. This 

35 technique is performed by a motion vector detector 19. Specifically, to obtain a difference between a signal stored in the 
frame memory 1 8 and an input signal, the detector 1 9 reads out the previous picture signal 21 from the frame memory 
18, and slide the signal 21 up and down, and right and left, to obtain differences, thereafter, select a slide quantity with 
which the difference becomes the smallest, as a motion vector 22, which is then transmitted to a decoding side by mul- 
tiplexing with an error signal 13 that has been multiplexed by a multiplexer 29a. The motion vector 22 is returned to the 

40 frame memory 18 as an address signal, to obtain a motion compensation predictive signal 20. The constitution of the 
encoder side is as mentioned above. 

In the decoder side, a transmitted difference signal 13 is decoded in the decorder 25 by means of inverse Huffman 
coding, inverse quantization, and inverse DCT and motion vector extraction, and the motion vector 22 is given to the 
motion compensation frame memory 18 to read the motion compensation predictive signal 20 obtained by sliding, from 

45 the frame memory 1 8. The difference signal 1 5 decoded in the decorder 25. and the signal 20 read out from the frame 
memory 18 are added in the adder 16 to generate a picture signal 17, i.e., a local decoded signal. Against the picture 
signal 17 thus encoded, decoded and reconstructed, a filtering using a postfilter 26 is performed. 
Alternatively, a filtering using a filter inside loop 23 is performed on the output from the frame memory 18, to reduce a 
block-shaped noise. Prior art techniques utilized in the above filterings will be described. 

so As a prior art 1 , there is discussed the technique disclosed in "Study of the Method for Improving the Image Quality 

of Block Encoding Picture by Post Filter Process" by Mitsuya et al. (TECHNICAL REPORT OF IEICE, IE84-46), as 
shown in Figure 22. In this technique, the filtering is limited to a block that has less brightness change of an obtained 
decoded picture, in other words, a block at whose boundary a block distortion is liable to be actualized. 
That is, a strong smoothing filtering is performed only on a pixel 50 on a block boundary. This filtering is performed by 

55 a low-pass filter (hereinafter referred to as "LPF") that takes a simple average of equal weight against nine pixels of a 
target pixel 40 and its surrounding eight pixels 41 to 48 in Figure 21 . In this! igure, numerals 51 and 52 designate a pixel 
inside a block and a block boundary, respectively. 

As a prior art 2, there is discussed the technique described in "Postprocessing Algorithms for Noise Reduction of 
MPEG Coded Video" by Nakajima et al. (TECHNICAL REPORT OF IEICE, IE94-7, DSP94-7 (1994-04), (hereinafter 
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referred to as "technique of Nakajima")). In the technique of Nakajima, the mixing ratio of a decoded pixel value and a 

simple average of its surrounding pixel values is changed appropriately depending on the local activity of a picture, 

quantizing parameter, block activity and the position relation with a block boundary. 

Assuming that a local region of the surrounding of the target pixel is a picture region (hereinafter referred to as 
5 "local region") of 3x3 pixels as shown in Figure 21 , and that a region (hereinafter referred to as "picture block")in which 

a b 2 (block activity) is computed is a region of 8x8 pixel in which DCT operation as shown in Figure 20 is performed, the 

quantizing parameter in decoding is stored for each picture block. 

In the local region as shown in Figure 20, a local average <d (x, y))is defined by the equation (1) for each of the 

entire pixels of a decoded picture. 
10 In taking the sum (I) of the local averages, if a pixel is outside the range of a picture region depending on the values 

of i and j, the value of a target pixel is employed instead of the pixel outside of the picture range. Note that d(x, y) is a 

decoded pixel value of the coordinate (x, y). 



+1 +1 



15 



<d(x,y)>=| £]Td(x-H,y+j) (1) 



i=-1j=-1 



Then, each pixel dispersion (hereinafter referred to as "local activity") as defined by the equation (2) is obtained in 
20 the local region. As the definition of the dispersion, the sum of the square the difference between the local region and 
the local average of the equation (1) is obtained in each pixel as in the equation (1). 



+1 +1 



25 i=-1j=-1 

Picture block dispersion (hereinafter referred to as "block activity") is defined by the equation (3). The block activity 
is defined for each of the picture blocks B 0 o to B22. as shown in Figure 20. 



30 



* bavg 2 (*, y) = E2> d 2 ( x b+K y b+1) (3) 



35 



64 

t=0 k=0 

wherein x b , y b are values of the upper left corner coordinate that belongs to the picture block number b n . 

An m(x, y), d(x, y) and <d(x, y) > are respectively, a corrected pixel value, a decoded pixel value and a simple aver- 
age pixel value of a local region, in the coordinate (x, y). 

40 m(x,y) = (1-p)<(x,y)>+pd(x,y) (4) 

Here, p is a mixing ratio of a decoded value and a value smoothed in the local region. 

45 2. x 2, x 

q d (x.y)-gn (x. y) 

<* d 2 (x.y) 

so In the pixel inside the picture block, 

-n 2 (x.y) = a bn 2 ^l (6) 
a d (x, y) 

55 

In the pixel on the picture block boundary, 
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a n 2 (x, y) = a bn 2 (7) 

In this case, a bn 2 is a functional value whose variable are quantizing parameter and block activity, and also the 
5 experimental value illustrated with the graph in the report. In accordance with the report of Nakajima, p in the above 
equation (4) is changed so as to increase the proportion of the simple average, in the event that various picture noises 
are liable to occur, that is, under the following conditions: 

a. when quantizing parameter is rough; 
10 b. when block activity is high; 

c. when noise occurs in a certain region of a picture; and 

d. when a sharp edge exists inside a picture block. 

As a prior art 3, a description will be given of the technique of "Gradient Inverse Weighted Smoothing Scheme and 
15 the Evaluation of its Performance" by D.C.C. Wang, A.H. Vagnucci and C.C.Li (Comp. Graphics Image Processing, Vol. 
15, pp.167-181, 1981) (hereinafter referred to as "technique of Wang"). The technique of Wang is one which takes 
weighted mean of a target pixel and its surrounding pixels. Its weighting factor is the inverse number of the absolute 
value of the difference between the target pixel and its surrounding pixels, and defined by the following equations (8) to 
(1 1), wherein d(x, y) is a decoded pixel value of the coordinate (x, y); and m(x, y) is a pixel value after taking weighted 
20 mean in the coordinate (x, y). 

+1 +1 

££w(i,j) d(x+i, y+j) 



25 



30 



35 



"<x. y) = l= - 1i °- 1 +1 +1 (8) 

ESw(i,j) 

i=-1j=-1 

where, 

AD y (x. y) = |d(x+i. y+j) <J(x, y)| (9) 



provided that when A {j (x, y)=0, AD ^x, y)=2, 



W(U) = A^) ™ 



40 (i,j = -1,0,+1) 

where 



45 



+1 +1 



w(0.0)=land SE^-i)*! < 11 > 



i=-i j=-i 



(i and j are not 0 at the same time) 
In this technique, if the difference between a target pixel and its surrounding pixels is small, the weighting factor 
so increases, so that the surrounding pixels are homogenized with a filtered target pixel value. On the contrary, if the dif- 
ference is great, i.e.. if an edge exists, the weighting factor decreases, leading to a filtering that preserves the edge. 

As a prior art 4, there will be discussed the technique of a filter utilized in ITU-T advice H. 261 (hereinafter referred 
to as "H. 261 standard") which was approved by World Telecommunication Standardization Commission. The above fil- 
ter is employed for an output of a frame memory in which a picture signal of the preceding frame is stored, that is. before 
55 obtaining a difference between the present picture signal and an input picture signal (see an in-loop filter 23 in Figure 
24). The output of the above frame memory is filtered by 8x8 picture blocks. 

• The above filter is constituted by operating a primary filter horizontally or vertically, that is, a non-recursive filter hav- 
ing coefficients of 1/4, 1/2. 1/4 against a pixel inside a picture block. In cases where a pixel employed in filter operation 
is outside the picture block, the above filter functions as a primary filter having coefficients of 0, 1 , 0. The filter coefficient 
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at the respective position of the 8x8 picture blocks are indicated in Figure 23, wherein numerals 61 , 62, 63 and 64 des- 
> ignate filter coefficients in four corners, up and down ends, right and left ends, and an inner region, respectively. For the 
up and down ends, and the left and right ends, there are six pixels respectively. In the inner region, there are 36 pixels. 
The filter coefficient is slided when applying to each pixel. 

5 In these basic filters of the prior art constitution, however, edge information that is inherently possessed by a picture 

except block noise is to be reduced because uniform filtering is performed on the entire decoded picture. In the case 
where only a block boundary is filtered using the technique of Mitsuya, when a continuous stroke originally crosses the 
block boundary, the continuous stroke blurs only on the block boundary. As a result, an observer will have an impression 
that such a block boundary is antifact. 

10 The technique of Nakajima utilizes a mixture of a simple average of a decoded pixel value and its surrounding pix- 
els values, so that its filtering blurs edge information within a picture block. Further, as defined in the equation (4), the 
proportion of a filter having no edge preserving property is enhanced on block boundaries, causing the same problem 
as in the technique of Mitsuya. 

In the technique of Wang having the edge preserving property, the weighting factor of a target pixel is fixed at 1/2, 

15 and therefore, plural processing are needed in the identical picture. Further, the replacement of a weighting factor when 
a pixel difference value AD jj is 0, may underestimate the weighting factor. Since this technique does not take into con- 
sideration block noise phenomenon on a block boundary, which occurs in diving a picture into blocks, a step of a block 
boundary might be erroneously recognized as an edge to be processed. 

The filter of the H. 261 standard also does not have the edge preserving property, causing a blur of a decoded pic- 

20 ture. 

SUMMARY OF THE INVENTION 

It is an object of the present invention to provide a picture signal smoothing device and its method that can reduce 
2$ the deterioration of a decoded picture while suppressing a decrease in resolution of the entire picture. 

It is a further object of the present invention to provide a picture encoding device and its method that can reduce 
deterioration of a decoded picture while suppressing a decrease in resolution of the entire picture. 

It is a further object of the present invention to provide a picture decoding device and its method that can reduce 
deterioration of a decoded picture while suppressing a decrease in resolution of the entire picture. 
30 Other objects and advantages of the present invention will become apparent from the detailed description given 
hereinafter; it should be understood, however, that the detailed description and specific embodiment are given by way 
of illustration only, since various changes and modifications within the scope of the invention will become apparent to 
the those skilled in the art from this detailed description. 

According to a first aspect of the present invention, a picture signal smoothing device that performs a smoothing 
35 process is performed by applying a smoothing filter in which a sampled and digitized picture signal is filtered by 
weighted mean of a target pixel and its surrounding pixels, includes means for detecting a difference between a target 
pixel and its surrounding pixels; means for taking the absolute value of the difference; means for adding a non-negative 
value (m) to the absolute value of the difference thus detected; means for taking the inverse number (weighting factor) 
of the value after the adding; means for multiplying the value after taking of the inverse number and the surrounding 
40 pixels values together; means for taking a sum (A) of the value after the multiplying; means for taking a sum (B) of the 
inverse number (the weighting factor) alone; and means for dividing the sum (A) by the sum (B) of the weighting factor 
and normalizing (A/B). 

Since the smoothing is performed by the inverse number of a difference value, if a target pixel is an isolated point, 
a difference value between the target pixel and each of its surrounding pixels is great, so that every surrounding has a 
45 similar weight. In the case where an edge exists near a target pixel, although the difference between the target pixel 
and the edge is great, the difference between the target pixel and its surrounding pixel except the edge is small. This 
leads to a constitution in which weighted mean is taken such that the weight of the edge side is small. A continuous 
edge can be preserved by this constitution, and the isolated point can be smoothed. In addition, by adding a non-neg- 
ative value, the degree of noise influence can be controlled in the operation, and there is no possibility that the operation 
so result after taking the inverse number is infinity. 

According to a second aspect of the present invention, the picture signal smoothing device of the first aspect 
includes means for diving a picture to be filtered into blocks; and means for appropriately changing a non-negative value 
to be added to the difference value between the target pixel and its surrounding pixels, according to the roughness of 
quantizing parameter of each block that is represented by quantizing characteristics utilized in encoding or decoding 
55 each block of a picture signal. 

Therefore, by controlling a non-negative value added prior to taking the inverse number in computing a weighting 
factor, using quantizing parameter, it is possible to erase noise that enlarges with enlarging quantizing error. 

According to a third aspect of the present invention, the picture signal smoothing device of the first aspect includes 
means for dividing a picture to be filtered into blocks; means for computing a block activity indicating a change degree 
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of a pixel by block; and means for appropriately changing a non-negative value to be added to a difference value 
between the target pixel and its surrounding pixels, according to the respective block activity. 

When a picture is flat and has low block activity, it is easy to observe noise. In such a case, however, it is possible 
to prevent noise from being recognized by enlarging a noise recognizable width and enhancing flat smoothing charac- 
5 teristics. 

According to a fourth aspect of the present invention, the picture signal smoothing device of the first aspect includes 
means for dividing a picture to be filtered into blocks; means for detecting whether a target pixel exists on a block bound- 
ary; means that if the target pixel exists on a block boundary, detects a block activity (adjacent block activity) of a block 
adjacent to the block boundary; and means for appropriately changing a non-negative value to be added to the differ- 
10 ence value between the target pixel and its surrounding pixels, according to the adjacent block activity. 

When a picture is adjacent to a flat block, on its boundary, a continuous edge of brightness (hue) often occurs. In 
such a case, therefore, by detecting that an adjacent block is flat and has low block activity, the smoothing characteris- 
tics of a point at which the flat block touches is enhanced to blur the step on the boundary. 

According to a fifth aspect of the present invention, the picture signal smoothing device of the first aspect includes 
15 means for dividing a picture to be filtered into blocks; means for mixing a signal value after the smoothing process by 
block, and a signal value prior to the smoothing process; means for appropriately changing the mixing ratio according 
to the roughness of quantizing parameter of each block that is represented by quantizing property utilized in encoding 
and decoding each block of a picture signal. 

Therefore, the constitution that the final filtering value of a target pixel is obtained by mixing the smoothed signal 
20 value and the signal value prior to the smoothing, allows to change the filtering characteristics by picture block, accord- 
ing to the respective pixel character. When a quantizing parameter is 1, no quantizing error occurs, and therefore, no 
further process is required against the value after the final filtering, that is, the signal prior to the smoothing can be uti- 
lized as it is. This leads to the constitution that the mixing ratio is controlled by quantizing parameter, and it is possible 
that a smoothed signal value is not utilized in the filtering. 
25 According to a sixth aspect of the present invention, the picture signal smoothing device of the first aspect includes 

means for dividing a picture to be filtered into blocks; means for mixing a signal value after the smoothing process by 
block, and a signal value prior to the smoothing process; means for appropriately changing the mixing ratio according 
to the respective block activity in a picture signal. 

Thus, for a flat block having low block activity, by making the final filtering value include more smoothed and flat sig- 
30 nal values, it is possible to erase noise that is noticeable in a flat block 

According to a seventh aspect of the present invention, the picture signal smoothing device of the first aspect 
includes means for dividing a picture to be filtered into blocks; means for mixing a signal value after the smoothing proc- 
ess by block, and a signal value prior to the smoothing process; means for detecting whether a target pixel exists on a 
block boundary; and means that if the target pixel exists on a block boundary, appropriately changes the mixing ratio 
35 according to an adjacent block activity of a block adjacent to the target pixel. 

When a picture is adjacent to a flat block having low block activity, on its boundary, a continuous edge of brightness 
(hue) often occurs. In such a case, therefore, by detecting that an adjacent block is flat, the flatness of filtering charac- 
teristics of a point at which the flat block touches is enhanced to blur the boundary. 

According to an eighth first aspect of the present invention, a picture signal smoothing device in which a filtering of 
40 a sampled and digitized picture signal is performed by a smoothing filter, includes means for dividing a picture to be fil- 
tered into blocks; means for detecting whether a target pixel exists on a block boundary; means for performing a filtering 
whose edge preserving property is weakened, if the target pixel exists on a block boundary; and means for performing 
a filtering having edge preserving property if the target pixel exists inside a block. 

Therefore, by enhancing the edge preserving property inside a block and weakening it on a block boundary, it is 
45 possible to perform a filtering that retains edge being important element of a picture signal as much as possible. 

According to a ninth aspect of the present invention, the picture signal smoothing device of the eighth aspect 
includes means for dividing a picture to be filtered into blocks; means for detecting whether a target pixel exists on a 
block boundary; means for computing the activity of the target pixel from the target pixel value and its surrounding pixels 
values; and means that if the target exists on a block boundary, appropriately changes the strength of edge preserving 
so property of a filter according to the activity of the target pixel. 

When a target pixel exists on a block boundary and its activity is high, a step often occurs on the block boundary. 
In such a case, therefore, by increasing the flatness of smoothing characteristics and the proportion of smoothing char- 
acteristics in filtering characteristics, such a step can be erased. 

According to a tenth aspect of the present invention, the picture signal smoothing device of the eighth aspect 
55 includes means for dividing a picture to be filtered into blocks; means for detecting whether a target pixel exists on a 
block boundary; means for detecting whether a block boundary on which the target pixel exists, continues horizontally 
or vertically; and means for appropriately changing the strength of edge preserving property according to the direction 
of the block boundary to which the target pixel belongs. 

Since a block noise occurs in the direction that a block boundary continues, by less preserving edge in that direc- 
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tion, it is possible to erase a block noise while strongly preserving an edge being vertical with respect to the block 
' boundary. 

According to an eleventh aspect of the present invention, a picture signal smoothing method in which a smoothing 
process is performed by applying a smoothing filter that performs a filtering of a sampled and digitized picture signal by 
5 weighted mean of a target pixel and its surrounding pixels, includes the steps of detecting a difference between a target 
pixel and its surrounding pixels; taking the absolute value of the difference; adding a non-negative value (m) to the 
absolute value of the difference thus detected; taking the inverse number (weighting factor) of the value after the adding; 
multiplying the value after taking of the inverse number and the surrounding pixels values together; taking a sum (A) of 
the value after the multiplying; taking a sum (B) of the inverse number (the weighting factor) alone; and dividing the sum 
10 (A) by the sum (B) of the weighting factor and normalizing (A/B). 

Since the smoothing is performed by the inverse number of a difference value, if a target pixel is an isolated point, 
a difference value between the target pixel and each of its surrounding pixels is great, so that the entire surrounding has 
a similar weight. In the case where an edge exist near a target pixel, although the difference between the target pixel 
and the edge is great, the difference between the target pixel and its surrounding pixel except the edge is small. This 
is leads to a constitution in which a weighted mean is taken such that the weight of the edge side is small. A continuous 
edge can be preserved by this constitution, and the isolated point can be smoothed. In addition, by adding a non-neg- 
ative value, the degree of noise influence can be controlled in the operation, and there is no possibility that the operation 
result after taking the inverse number is infinity. 

According to a twelfth aspect of the present invention, the picture signal smoothing method of the eleventh aspect 
20 includes the steps of diving a picture to be filtered into blocks; and appropriately changing a non-negative value to be 
added to the difference value between the target pixel and its surrounding pixels, according to the roughness of quan- 
tizing parameter in each block, which is represented by quantizing characteristics utilized in encoding and decoding 
each block of a picture signal. 

Therefore, by controlling a non-negative value that is added prior to taking the inverse number in computing a 
25 weighting factor, using quantizing parameter, it is possible to erase noise that enlarges with enlarging quantizing error. 

According to a thirteenth aspect of the present invention, the picture signal smoothing method of the eleventh 
aspect includes the steps of dividing a picture being to be filtered into blocks; computing a block activity indicating a 
change degree of a pixel, by block; and appropriately changing a non-negative value to be added to a difference value 
between the target pixel and its surrounding pixels, according to the respective block activity. 
30 When a picture is flat and has low block activity, it is easy to observe noise. In such a case, however, it is possible 
to prevent noise from being recognized by enlarging a noise recognizable width and enhancing flat smoothing charac- 
teristics. 

According to a fourteenth aspect of the present invention, the picture signal smoothing method of the eleventh 
aspect includes the steps of dividing a picture to be filtered into blocks; detecting whether a target pixel exists on a block 
35 boundary; if the target pixel exists on a block boundary, detecting a block activity (adjacent block activity) of a block adja- 
cent to the block boundary; appropriately changing a non-negative value to be added to the difference value between 
the target pixel and its surrounding pixels, according to the adjacent block activity. 

When a picture is adjacent to a flat block, on its boundary, a continuous edge of brightness (hue) often occurs. In 
such a case, therefore, by detecting that an adjacent block is flat having low block activity, the flatness of the smoothing 
40 characteristics of a point at which the flat block touches is enhanced to blur the step on the boundary. 

According to a fifteenth aspect of the present invention, the picture signal smoothing method of the eleventh aspect 
includes the steps of dividing a picture to be filtered into blocks; mixing a signal value after the smoothing process by 
block, and a signal value prior to the smoothing process; and appropriately changing the mixing ratio according to the 
roughness of quantizing parameter of each block, which is represented by quantizing characteristics utilized in encod- 
45 ing and decoding each block of a picture signal. 

Therefore, the constitution in which the final filtering value of a target pixel is obtained by mixing the smoothed sig- 
nal value and the signal value prior to the smoothing, allows to change the filtering characteristics of each picture block, 
according to the respective pixel character. When the quantizing parameter is 1 , no quantizing error occurs, and there- 
fore, no further process is required against the value after the final filtering, that is, the previous signal can be utilized 
so as it is. This leads to the constitution that the mixing ratio is controlled by the quantizing parameter, and it is possible 
that a smoothed signal value is not utilized in the filtering. 

According to a sixteenth aspect of the present invention, the picture signal smoothing method of the eleventh 
aspect includes the steps of dividing a picture being to be filtered into blocks; mixing a signal value after the smoothing 
process by block, and a signal value prior to the smoothing process; and appropriately changing the mixing ratio 
55 according to the respective block activity in a picture signal. 

Thus, for a flat block having low block activity, by making thef inal filtering value include more smoothed and flat sig- 
nal values, it is possible to erase noise being noticeable in a flat block. 

According to a seventeenth aspect of the present invention, the picture signal smoothing method of the eleventh 
aspect includes the steps of dividing a picture to be filtered into blocks; mixing a signal value after the smoothing proc- 
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ess by block, and a signal value prior to the smoothing process; detecting whether a target pixel exists on a block 
boundary; and if the target pixel exists on a block boundary, appropriately changing the mixing ratio according to an 
adjacent block activity of an adjacent block that touches the target pixel. 

When a picture is adjacent to a flat block having low block activity, on its boundary, a continuous edge of brightness 
(hue) often occurs. In such a case, therefore, by detecting that an adjacent block is flat, the flatness of the filtering char- 
acteristics of a point at which the flat block touches is enhanced to blur the boundary. 

According to an eighteenth aspect of the present invention, a picture signal smoothing method in which a filtering 
of a sampled and digitized picture signal is performed by a smoothing filter, includes the steps of dividing a picture to 
be filtered into blocks; detecting whether a target pixel exists on a block boundary; performing a filtering whose edge 
preserving property is weakened, if the target pixel exists on a block boundary; and performing a filtering having edge 
preserving property if the target pixel exists inside a block. 

Therefore, by enhancing edge preserving property inside a block and weakening it on a block boundary, it is pos- 
sible to perform a filtering that retains edge being important element of a picture signal as much as possible. 

According to a nineteenth aspect of the present invention, the picture signal smoothing method of the eighteenth 
aspect includes the steps of dividing a picture to be filtered into blocks; detecting whether a target pixel exists on a block 
boundary; computing an activity of the target pixel from the target pixel value and its surrounding pixels values; and 
appropriately changing the strength of edge preserving property of a filter according to the activity of the target pixel, if 
the target pixel exists on a block boundary. 

When a target pixel exists on a block boundary and its activity is high, a step often occurs on the block boundary. 
In such a case, therefore, by increasing the flatness of smoothing characteristics and the proportion of smoothing char- 
acteristics in filtering characteristic, such a step can be erased. 

According to a twentieth aspect of the present invention, the picture signal smoothing method of the eighteenth 
aspect includes the steps of dividing a picture to be filtered into blocks; detecting whether a target pixel exists on a block 
boundary; detecting whether a block boundary at which the target pixel exists, continues horizontally or vertically; and 
appropriately changing the strength of edge preserving property according to the direction of the block boundary to 
which the target pixel belongs. 

Since a block noise occurs in the direction that a block boundary continues, the employment of a filter that weakly 
preserves the edge in that direction, allows to erase a block noise while strongly preserving the edge being vertical with 
respect to the block boundary. 

According to a twenty-first aspect of the present invention, a picture encoding device in which a filter is attached to 
a decoding loop having a frame memory for storing a signal corresponding to one or a plurality of frames or fields, and 
an inter-frame encoding is performed by a corresponding pixel between an input signal and the frame memory, is char- 
acterized in that a filter having edge preserving property is employed as the above filter, and a picture signal after being 
processed in a picture signal smoothing device is input, as a local decoding signal, to the frame memory storing a pic- 
ture signal of the previous frame. 

Thus, the construction that a pixel value after the filtering is written to a decoded signal in the frame memory, allows 
to reduce a block noise while retaining edge, to decrease predictive error in input picture signal between the present 
frame and the next frame, and to reduce transmission bit quantity. 

According to a twenty-second aspect of the present invention, the picture encoding device of the twenty-first aspect 
includes means for dividing a picture into blocks and computing an inter-frame difference by block; means for determin- 
ing whether the difference exceeds a certain threshold value; and means that only when the difference exceeds the 
threshold value, the corresponding block is processed by a filter having edge preserving property. 

When there exists a picture block to which no additional difference component is added (i.e., a picture block signal 
that has been filtered and stored in the frame memory), by not making another filtering against the identical picture, a 
picture blur can be prevented. 

According to a twenty-third aspect of the present invention, a picture encoding device in which a filter is attached 
to a decoding loop having a frame memory for storing a signal corresponding to one or a plurality of frames or fields, 
and an inter-frame encoding is performed by a corresponding pixel between an input signal and the frame memory, 
includes means for computing a motion vector; means for reading a picture value stored in the frame memory with con- 
sideration of the motion vector; and a filter having edge preserving property, which filter processes an output signal of 
the frame memory 

Therefore, the feature that an in-loop filter inside an encoding loop is constituted by the above filter, enables to 
decrease predictive error in input picture signal between the present frame and the next frame, and to reduce transmis- 
sion bit quantity. 

According to a twenty-fourth aspect of the present invention, a picture decoding device in which a filter is attached 
to a decoding loop having a frame memory for storing a signal corresponding to one or a plurality of frames or fields, 
and an inter-frame decoding is performed by a corresponding pixel between an input signal and the frame memory, is 
characterized in that a filter having edge preserving property is employed as the above filter, and a picture signal after 
being processed in a picture signal smoothing device is input, as a local decoding signal, to the frame memory storing 
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the picture signal of the previous frame. 

Therefore, the construction that a pixel value after the filtering is written to a decoded signal in the frame memory, 
allows to reduce a block noise while retaining edge, to decrease predictive error in input picture signal between the 
present frame and the next frame, and to reduce transmission bit quantity. 

s According to a twenty-fifth aspect of the present invention, the picture decoding device of the twenty-fourth aspect 

includes means for dividing a picture into blocks and computing an inter-frame difference by block; means for determin- 
ing whether the difference exceeds a certain threshold value; and means that only when the difference exceeds the 
threshold value, the corresponding block is processed by a filter having edge preserving property. 

In the case where there exists a picture block to which no additional difference component is added (i.e., a picture 

10 block signal that has been filtered and stored in the frame memory), by not performing another filtering against the iden- 
tical picture, a picture blur can be prevented. 

According to a twenty-sixth aspect of the present invention, a picture decoding device in which a filter is attached 
to a decoding loop having a frame memory for storing a signal corresponding to one or a plurality of frames or fields, 
and an inter-frame decoding is performed by a corresponding pixel between an input signal and the frame memory, 

15 includes means for computing a motion vector; means for reading a picture value stored in the frame memory with con- 
sideration of the motion vector; and a filter having edge preserving property, which filter processes an output signal of 
the frame memory. 

Therefore, the feature that an in-loop filter inside an encoding loop is constituted by the above filter, enables to 
decrease predictive error in input picture signal between the present frame and the next frame, and to realize a picture 

20 signal decoding that reduces transmission bit quantity. 

According to a twenty-seventh aspect of the present invention, a picture encoding method in which a filter is 
attached to a decoding loop having a frame memory for storing a signal corresponding to one or a plurality of frames or 
fields, and an inter-frame encoding is performed by a corresponding pixel between an input signal and the frame mem- 
ory, includes the steps of performing a process using a filter having edge preserving property; and inputting a picture 

25 signal after being processed in a picture signal smoothing device to the frame memory storing a picture signal of the 
previous frame, as a local decoding signal. 

Therefore, the feature that an in-loop filter inside an encoding loop is constituted by the above filter, enables to 
decrease a block noise while retaining edge, to decrease predictive error in input picture signal between the present 
frame and the next frame, and to reduce transmission bit quantity. 

30 According to a twenty-eighth aspect of the present invention, the picture encoding method of the twenty-seventh 
aspect includes the step of dividing a picture into blocks and computing an inter-frame difference by block; the step of 
determining whether the difference exceeds a certain threshold value; and the step in which only when the difference 
exceeds the threshold value, such a block is processed by a filter having edge preserving property. 

In the case where there exists a picture block to which no additional difference component is added (i.e., a picture 

35 block signal that has been filtered and stored in the frame memory), by not performing another filtering against the iden- 
tical picture, a picture blur can be prevented. 

According to a twenty-ninth aspect of the present invention, a picture encoding method in which a filter is attached 
to a decoding loop having a frame memory for storing a signal corresponding to one or a plurality of frames or fields, 
and an inter-frame encoding is performed by a corresponding pixel between an input signal and the frame memory, 

40 includes the steps of computing a motion vector; reading a picture value stored in a frame memory with consideration 
of a motion vector; and processing an output signal of the.frame memory by using a filter having edge preserving prop- 
erty. 

Therefore, the feature that an in-loop filter inside an encoding loop is constituted by the above filter, enables to 
decrease predictive error in input picture signal between the present frame and the next frame, and to reduce transmis- 
45 sion bit quantity. 

According to a thirtieth aspect of the present invention, a picture decoding method in which a filter is attached to a 
decoding loop having a frame memory for storing a signal corresponding to one or a plurality of frames or fields, and an 
inter-frame decoding is performed by a corresponding pixel between an input signal and the frame memory, includes 
the steps of performing a process using a filter having edge preserving property; and inputting a picture signal after 
so being processed in a picture signal smoothing device to the frame memory storing the picture signal of the previous 
frame, as a decoded signal. 

Therefore, the construction that a pixel value after the filtering is written to a decoded signal in the frame memory, 
allows to reduce a block noise while retaining edge, to decrease predictive error in input picture signal between the 
present frame and the next frame, and to reduce transmission bit quantity. 
55 According to a thirty-first aspect of the present invention, the picture decoding method of the thirtieth aspect 
includes the step of dividing a picture into blocks and computing an inter-frame difference by block; the step of deter- 
mining whether the difference exceeds a certain threshold value; and the step that only when the difference exceeds 
the threshold value, the corresponding block is processed by a filter having edge preserving property. 

In the case where there exists a picture block to which no additional difference component is added (i.e., a picture 
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block signal that has been filtered and stored in the frame memory), by not performing another filtering against the iden- 
tical picture, a picture blur can be prevented. 

According to a thirty-second aspect of the present invention, a picture decoding method in which a filter is attached 
to a decoding loop having a frame memory for storing a signal corresponding to one or a plurality of frames or fields, 
5 and an inter-frame decoding is performed by a corresponding pixel between an input signal and the frame memory, 
includes the steps of computing a motion vector; reading a picture value stored in the frame memory with consideration 
of the motion vector; and processing an output signal of -the frame memory by using a filter having edge preserving 
property. 

Therefore, the features that an in-loop filter inside an encoding loop is constituted by the above filter, enables to 
to decrease predictive error in input picture signal between the present frame and the next frame, and to reduce transmis- 
sion bit quantity. 

BRIEF DESCRIPTION OF THE DRAWINGS 

15 Figure 1 is a block diagram illustrating an inter-frame encoding system to which the present invention is applied. 

Figure 2 is a block diagram illustrating a constitution of a smoothing filter of the present invention. 

Figure 3 is a block diagram of a further constitution that is added to the smoothing filter of Figure 2. 

Figure 4 is a block diagram of a further constitution that is added to the smoothing filter of Figure 2. 

Figure 5 is a block diagram of a further constitution that is added to the smoothing filter of Figure 2. 
so Figure 6 is a block diagram of a further constitution that is added to the smoothing filter of Figure 2. 

Figure 7 is a block diagram of a further constitution that is added to the smoothing filter of Figure 2. 

Figure 8 is a block diagram of a further constitution that is added to the smoothing filter of Figure 2. 

Figure 9 is a block diagram of a constitution of the smoothing filter of the present invention. 

Figure 1 0 is a block diagram of a further constitution of the smoothing filter of the present invention. 
25 Figure 1 1 is a block diagram of a further constitution of the smoothing filter of the present invention. 

Figure 1 2 is a block diagram of a further constitution of the smoothing filter of the present invention. 

Figure 13 is a detail flow chart of a filter controlling of the present invention. 

Figure 1 4 is a detail flow chart of a further filter controlling of the present invention. 

Figure 1 5 is a detail flow chart of a further filter controlling of the present invention. 
30 Figure 1 6 is a detail flow chart of a further filter controlling of the present invention. 

Figure 1 7 is a detail flow chart of a further filter controlling of the present invention. 

Figure 18 is a detail flow chart of a further filter controlling of the present invention. 

Figure 19 is a schematic flow chart of a further filter controlling of the present invention. 

Figure 20 is a diagram illustrating block dividing and adjacent blocks adjacent to a target pixel. 
35 Figure 21 is a diagram illustrating local regions of the equations (1), (2) and (7). 

Figure 22 is a diagram explaining one example of IE84-46 in the TECHNICAL REPORT OF IEICE. 

Figure 23 is a diagram explaining filter coefficients of the H. 261 standard. 

Figure 24 is a block diagram of an inter-frame encoding system to which a prior art filtering system is applied. 
40 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

A description will be given of embodiments of the present invention, referring to Figure 1 through Figure 20. 
[Embodiment 1] 

45 

In order to reduce deterioration of a decoded picture and suppress a decrease of resolution of the entire picture, 
the embodiment 1 of the present invention has a constitution in which after adding a non-negative value to a difference 
value between a decoded target pixel and its surrounding pixel, an inverse number is taken and, there is employed a 
smoothing filter that utilizes such an inverse number as a weight and has edge preserving property. In this smoothing 

so filter, a smoothing is performed by the inverse number of a difference value, and therefore, if a target pixel is an isolated 
point, the difference between the target pixel and every surrounding pixel is great and all surrounding pixels have a sim- 
ilar weight. If there is an edge adjacent to a target pixel, a difference between the target pixel and the edge is great, but 
the difference between the target pixel and the surrounding pixel except the edge is small. This leads to a constitution 
in which a weighted mean is taken so that the weight of the edge side is small. A continuous edge can be preserved by 

55 this constitution, and the isolated point can be smoothed. By controlling a non-negative value that is added prior to tak- 
ing the inverse number in computing a weighting factor, it is possible to erase noise of not more than a certain value, as 
well as quantizing error. Further, the use of the above smoothing filter enables to greatly reduce distortion occurred on 
block boundaries, leading to much visual improvement of picture quality. Furthermore, since in the smoothing process, 
characteristics of the smoothing filter is changed according to the roughness of a decoded parameter and the activity 
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of a picture signal, it is possible to suppress deterioration of resolution of the entire decoded picture, to obtain a decod- 
'* ing picture that is visually legible and has high quality. 

Figure 1 is a block diagram of a picture encoding and decoding device in the present invention. There are indicated 
the positions of an in-loop filter 23 and a postfilter 26, in addition to the constitution as previously described in the prior 
s art. 

Figure 2 shows a constitution of a block of a smoothing filter that is installed in the in-loop filter 23 or the postfilter 
26 in Figure 1 . A difference circuit 231 detects a difference between a target pixel value and its surrounding pixel value, 
and an absolute value circuit 232 takes the absolute value of the detected difference. The absolute value is added to a 
non-negative value m in an adder 233, and the inverse number of a resultant value thus added is taken in an inverse 
10 number circuit 234, to obtain a weighting factor. The weighting factor is multiplied by the surrounding pixel value in a 
multiplier 235. A sum circuit 236 takes the sum of the multiplied values. A sum circuit 237 takes the sum of the weighting 
factors. The sum taken in the sum circuit 236 is divided by the sum taken by the sum circuit 237, to obtain a smoothed 
pixel value. 

Figure 3 shows a constitution added to the smoothing filter in Figure 2. A picture block divider 2331 divides a picture 
75 to be filtered into blocks. A quantizing parameter for each picture of a block thus divided is computed in a quantizing 
parameter computer 2332. According to roughness of the quantizing parameter, a non-negative added value variant 
part 2333 changes a non-negative value m to be added in the adder 233. 

Figure 4 shows a further constitution to be added to the smoothing filter in Figure 2. A block activity computer 2334 
computes an activity of each picture in a divided block. According to the block activity, the non-negative added value 
20 variant part 2333 changes a non-negative value m to be added in the adder 233 in Figure 2. 

Figure 5 shows a further constitution to be added to the smoothing filter in Figure 2. A pixel position detector 2335 
detects whether a target pixel exists on a block boundary. If the target pixel exists on the block boundary, the block activ- 
ity computer 2334 detects the activity of a block adjacent to the target pixel, and the non-negative added value variant 
part 2333 changes a non-negative value m to be added in the adder 233 in Figure 2. 
25 Figure 6 shows a further constitution to be added to the smoothing filter in Figure 2. The quantizing parameter com- 

puter 2332 computes a quantizing parameter of each block in a picture signal. According to roughness of the respective 
quantizing parameter, a signal mixer 2336 appropriately changes the mixing ratio of a signal value after being smoothed 
by block in a picture signal, and a signal value before being smoothed. 

Figure 7 shows a further constitution to be added to the smoothing filter in Figure 2. The block activity computer 
30 2334 computes the activity of each block in a picture signal. According to the respective activity, the signal mixer 2336 
appropriately changes the mixing ratio of a signal value after being smoothed by block in a picture signal, and a signal 
value before being smoothed. 

Figure 8 shows a further constitution to be added to the smoothing filter in Figure 2. The pixel position detector 
2335 detects whether a target pixel exists on a block boundary. If the target pixel is in such a position that it contacts 
35 with the block boundary, the signal mixer 2336 appropriately changes the mixing ratio of a signal value after being 
smoothed by block in a picture signal, and a signal value before being smoothed. 

Figure 9 shows a constitution of the smoothing filter in Figure 2. The pixel position detector 2335 detects whether 
a target pixel exists on a block boundary. If the target pixel exists on the block boundary, there is selected a filter circuit 
2337 that performs a smooth filtering whose edge preserving property is weakened. If the target pixel exists inside a 
40 block, there is selected a filter circuit 2338 that performs a smooth filtering having edge preserving property. 

Figure 10 shows a further constitution of the smoothing filter in Figure 2. The pixel position detector 2335 detects 
whether a target pixel exists on a block boundary. If the target pixel exists on the block boundary, the activity computer 
2340 computes the activity of the target pixel from the target pixel value and its surrounding pixel value. According to 
the activity, the filter circuit 2337 that performs a smooth filtering whose edge preserving property is weakened, or the 
45 filter circuit 2338 that performs a smooth filtering having edge preserving property is selected. 

Figure 1 1 shows a further constitution of the smoothing filter in Figure 2. The pixel position detector 2335 detects 
whether a target pixel exists on a block boundary. If the target pixel exists on the block boundary, a boundary continuous 
direction detector 2341 detects whether the block boundary to which the target pixel belongs continues horizontally or 
vertically. According the direction of the block to which the target pixel belongs, the filter circuit 2337 that performs a 
so smooth filtering whose edge preserving property is weakened, or the filter circuit 2338 that performs a smooth filtering 
having edge preserving property is selected. 

Figure 1 2 shows a constitution in which the picture encoding and decoding device in Figure 1 or the smoothing filter 
in Figure 2 through Figure 1 1 is realized by software. A picture input part 300 inputs a picture data. The input picture 
data is input directly to a CPU 1 00 via a bus 1 50. alternatively, after the input picture data is stored in a picture memory 
55 200. it is read from the picture memory 200 and input to the CPU 100. The picture data input to the CPU 100 is proc- 
essed by software and input directly to a picture output part 400 via the bus 150, alternatively, after the input picture 
data is stored in the picture memory 200, it is read from the picture memory 200 and input to the picture output part 400. 

In order to clarify the definition of the technique of the embodiment 1 of the present invention, similar equations as 
defined in the description of the prior art will be defined here. 
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Assuming that a local region of the surrounding of a target pixel (hereinafter referred to as "local region") is a picture 
region of 3x3 pixels as shown in Figure 21 , and that a region in which a b 2 (block activity) is computed is a region (here- 
inafter referred to as "picture block") of 8x8 pixels in which DCT operation as shown in Figure 20 is performed, a quan- 
tizing parameter in decoding is stored by picture block. 

Against each of the entire pixels of a decoded picture, a local average value <d(x, y) > is defined in a local region as 
shown in Figure 21 , based on the equation (1 2). 

In taking the sum z of the local average values, if the value is outside of the range of the picture region depending 
on the values of i and j, the value of the target pixel is employed. Note that d(x, y) is a decoded pixel value of the coor- 
dinate (x, y). 



( *y)) = ^I d ( x+i -y+i) d2) 

l=-1j=-1 



Then, a dispersion of each pixel (hereinafter referred to as "local activity"), as defined by the equation (13), is 
obtained in the local region. Here, the sum of the square of the difference between the local region and the local aver- 
age of the equation (12) is obtained as the definition of the dispersion. This definition is given to every pixel as in the 
equation (12). 



+1 +1 

d 2 ( x > v ) - § Z I>( x+i ' V+j)-<d(x, y) )) 2 (13) 



Picture block dispersion (hereinafter referred to as "block activity") is defined by the equation (14). The block activity 
is defined by block, as shown in Figure 20. 

+7+7 

a b 2 IM = ^ EE* d 2 <x b +K Yb+1) (14) 

t=0k=0 



wherein x b , y b are values of the upper left corner coordinate that belongs to the picture block number b n . 
35 An m(x, y), d(x, y) and (s(x, y) ) are respectively a corrected pixel value, a decoded pixel value and a pixel value that 
has been smoothed by local region. 

m(x, y) = (1 -X) d(x. y) + A,s(x, y) (15) 

40 Here, X is a mixing ratio of a decoded value and a value smoothed by local region. The mixing ratio is defined by 
the following equations (1 6) to (1 8), wherein QP is a quantizing parameter in decoding, a b 2 is a block activity computed 
by the equation (14). 

If the quantizing parameter is great, quantizing error is liable to occur. If the block activity of a decoded picture is 
great, many steps of a pixel value present. For this, it is so constituted that the smoothing characteristics is enhanced. 

45 Thus it is possible to change filtering characteristics by picture block, depending on each pixel character. Addition- 
ally, when the quantizing parameter is 1 , no quantizing error occurs, so that the value after the final filtering can be held 
at a signal prior to the smoothing process. As a result, the constitution in which the mixing ratio is controlled by quan- 
tizing parameter, enables to prevent a smoothed signal value from being utilized in the filtering, allowing to skip the fil- 
tering process. Therefore, when the device is constituted by CPU or DSP, its processing time can be reduced. 

so In a flat block having low block activity, by making the final filtering value contain smoothed and flat signals as much 

as possible, it is possible to erase noise being noticeable in aflat block. 
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A decoded value d(x, y) in the coordinate (x, y) is not included in the smoothing of the second term s(x, y) in the 
right side as defined in the equation (15). From the definitions of the equations (1 8) and (21 ), the pixel value of a target 
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position d(x, y) is not included in the pixel value s(x, y) after the smoothing. Therefore, the range of the value X is as set 
* below. 

0 = ^ = ^max O 7 ) 

5 

When X of the equation (16) results in underflow with a value in the range of the equation (17), it is fixed at the min- 
imum value. In overflow, it is fixed at the maximum value. 

Next defined is a pixel value computing of a smoothing of the second term in the right side as employed in the equa- 
tion (15). 

10 The smoothing is to take a weighted mean of a surrounding pixel according to a weight as defined in the equation 
(21). Since the weight is defined by the inverse number of a difference between pixel values, a place such as an edge 
in which the difference between pixel values is great, has no influence on the smoothing. That is, the smoothing is so 
performed that edge is preserved. 

75 +i +1 

ZZ w 0. i) d ( x+i ' y + i) 
s(x. y) = i= - 1|= - 1 , 1 +1 (18) 

SEw(ij) 

20 U-1J-1 

A difference from a surrounding pixel of the coordinate (x, y) is taken by local region and, for normalization, a non- 
negative number m is added to the absolute value of the difference from each pixel. By the action of this term, even if 
d(x, y)-d(x+i. y+j)=0 , there is no possibility that the equation (21) becomes infinity. 
25 Further, when the difference value from the surrounding pixel is m or less, by the action of this term, a value near 

a simple average of the surrounding pixels is to be computed. 

Thus, since the smoothing is performed by the inverse number of the difference value as previously described, a 
continuous edge can be preserved by this constitution, and an isolation point can be smoothed. Furthermore, by adding 
a non-negative value, it is possible that the operation result after taking the inverse number will be infinity. 



30 



40 



AD B (x, y) = |d(x+1 , y+j)-d(x, y)|+m (19) 



If the value m of the second term in the right side is great, there functions to mask a pixel value difference (the first 
term in the right side) being the edge preserving effect. Although the mask value may be replaced with a fixed value, in 
35 order that the change of quantizing error is reflected, it is controlled by the quantizing parameter as indicated in the 
equation (20). This enables to erase noise that enlarges with enlarging the quantizing error. 

m=Q^l (20) 



(21) 



45 (i, j = -1 , 0, +1 ; and w(0, 0) = 0) 

When block distortion occurs, the local activity 6 b 2 is 0, inside a block. A pixel on a block boundary has a great local 
activity because a surrounding pixel touches an adjacent block to cause a step. In such circumstances, however, since 
an average is taken by block, the block activity 6 b 2 is below a certain threshold value 5 2 - ,n order to strongly erase a 
block distortion, an additional limiting item is set to X being computed by the equations (16) and (1 7). This enables to 
so enlarge a noise recognizable width and enhance flat smoothing characteristics, thereby preventing noise from being 
recognized. 
When 

a b 2 [bn]=i 8 2 . m=QP[bn], X=X max (22) 

55 

If a target pixel exists on a block boundary and is under the following conditions of (i) and (ii). the smoothing char- 
• acteristics is enhanced. These conditions leads to such a filtering that the edge preserving property is strong inside a 
block and is weak on a block boundary. 

Therefore, if a picture is adjacent to a flat block, it is detected that such an adjacent block is flat and has low block 
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activity, so that the flatness of the smoothing characteristics of a point at which the flat block touches, is increased to 
blur a predictable boundary step. 

If a picture is adjacent to a flat block having low block activity, it is detected that such an adjacent block is flat, so 
that the flatness of the smoothing characteristics of a point at which the flat block touches, is increased to blur a predict- 
able boundary step. 

If a target point exists on a block boundary and has high activity, the flatness of the smoothing characteristics and 
the proportion of the smoothing characteristics in filtering characteristics are increased to erase a predictable step. 

m = QP[bn],X = X max (23) 

(i) In the case where a block activity of a block adjacent to a picture block to which a target pixel touches is lower 
than a certain threshold value 8 2 . 

In the equation (22), only an activity of a block to which a target pixel belongs is determined. The conditions (i) 
determines an activity of an adjacent block. 
is In Figure 20. provided that the number 30 is a target pixel, the number 31 is an adjacent block. 

(ii) In the case where a local activity of a target pixel is higher than a certain threshold value 8 3 . This corresponds 
to the case where a waveform that is composed of a frequency component having a high rectangular conversion 
(DCT conversion), is cut on a block boundary, and a step occurs between the waveform and an adjacent block. 

Each example of the threshold values X max , 8 1 , 8 2 and 8 3 is as below. 
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X max = 0- 7 5 (24) 



Of . 



8 l = 2 f XO Pic (25) 

8 2 = 75 for a brightness (Y) 

8 2 = 30 for a hue (U, V) (26) 

8 3 = 35 x 8 pic (27) 



35 pic - XZ ct b 2 fbn] (28) 

M N 



wherein a ^ is a mean of block activities in the entire picture; M is the block number in the horizontal direction; and N 
is the block number in the vertical direction. 
40 when a picture is divided into blocks of 8x8, x of the coordinate (x, y) is a multiple of 8, or a surplus of the mul- 

tiple of 8 is 7, there exists a block boundary that continues in the vertical direction. 

Similarly, when y is a multiple of 8, or a surplus of the multiple of 8 is 7, there exists a block boundary that con- 
tinues in the horizontal direction. 

(iii) In the case where a target pixel belongs to a block boundary, and its local activity is higher than 8 3 
45 Since it is determined that a block noise occurs on a block boundary, the above equations (19) and (20) should 

be newly defined as follows: 

AD 5 (x, y)= |d(x+1 , y+j)-d(x, y)|+m(i, j) (29) 

50 m(i, j) = QP[bn] (i, j = -1 . 0, +1) (30) 

When a coordinate where the block boundary that continues in the horizontal direction, is under the above con- 
ditions (iii), a horizontal block noise occurs, to weak smoothing characteristics of a pixel horizontally adjacent to a 
target pixel. 
55 When 

i=0; and j=-1, +1, m(0, j)=2xQP[bn] (31) 

Similarly, for a vertical block noise: 
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i=-1 , +1 ; and j=0, m(i, 0)=2xQP[bn] (32) 

5 The above constitution underestimates weighting factors of two pixels facing each other across a target pixel being 

parallel to a block noise. This leads to a strong function of filter characteristics in the vertical direction with respect to 
the block noise direction. This enables to erase block noise and strongly preserve an edge being vertical with respect 
to the direction of a block boundary. As a result, it is possible to perform such a filtering that retains edge being impor- 
tant element of a picture signal as much as possible. 
io The operation of the filter 23 or 26 in Figure 1 will be described in accordance with flow charts of Figures 1 3 through 
Figure 18, which illustrate the actual computing procedure for the device as shown in Figure 12. 

In Step SI , against every pixel of a picture signal, a local mean <d(x, y) ) for each pixel of a decoded picture is com- 
puted based on the equation (12). 

In Step S2, similarly to S1 , a local activity a d 2 (x t y) is computed based on the equation (1 3). 
75 In Step S3, a block activity a t^fan] f° r eacn picture block of 8x8 dot is computed based on the equation (1 4), and 
a quantizing parameter QP[bn] is read in. 

In Steps S4, S5 and S25, a loop is turned on every picture block. In each loop, the following calculations is made 
on each block. 

In Step S6, a mixing ratio X is computed based on the equation (16). At this time, 6 1 is a fixed value. 
20 In Step S7, a non-negative value m is determined based on the equation (20). 

In Steps S8 and S9, if a mixing ratio X exceeds a limit range, the X is limited to X max (see the equation (1 7)). 

In Steps S1 0 and S1 1 , if the block activity cr t> 2 [bn] of a picture block as computed in S3 is not more than a certain 
threshold value 6 2 , '* is replaced with X=X max , m=QP[bn] , as indicated in the equation (22). 

In Steps S12, S13, S14, S21, S22, S23 and S24, a loop is turned on every pixel in a picture block. Variables k and 
25 I are incremental in x and y directions, respectively. Since the size of the picture block is 8x8, there increases in a range 
of 0 to 7. Each of Xt, and y b is the upper left corner coordinate in a target pixel block. 

In Step S1 6. it is determined whether when the variables k and I are 0 or 7, a target pixel exists on a block boundary 
as shown by the circles in Figure 20. 

In Steps S26, S27 and S28, when the variable I is 0, the target pixel is at the upper side of a target block in 
30 Figure 20. Therefore, if a block B 0 i adjacent to the upper side of the target block is a b 2 [bn-M] ^ 5 2 , there is replaced 
with X=X max , m=QP[bn] (see the equation (23) and the conditions (i)). 

In Steps S29, S30 and S31, when the variable k is 7, the target pixel is at the right side of the target block B 1V 
Therefore, a block B 12 adjacent to the right of the target block is a b 2 [bn+1]^ 6 2 , there is replaced with 
X=X max> m=QP[bn] 

35 In Steps S32, S33 and S34, when the variable 1 is 7, the target pixel is at under side of the target block B 1 -j . There- 
fore, if a block B 21 adjacent to the bottom of the target block is a b 2 [bn+M]^ 6 2 , there is replaced with 
X=X max , m=QP[bn] 

In Steps S35, S36 and S37, when the variable k is 0, the target pixel is at the left side of the target block B-j 1 . There- 
fore, if a block B 10 adjacent to the bottom of the target block is a b 2 [bn-1]^6 2 , there is replaced with 
40 max. m=QP[bn] . 

In Steps 38 and S39, when the local activity a d 2 (x, y) is not less than the threshold value 8 3 , there is replaced with 
X-X max , m=QP[bn] (see the equation (23) and the condition (ii)). 

In Step 1 7, setting the variable i and j be ±1 , a difference between a target pixel and its surrounding pixel is com- 
puted, and a non-negative value m is added to the absolute value of the difference (see the equation (1 9)). The inverse 
45 number of the equation (19) is employed as a weight w(i, j) of the target pixel (see the equation (21)). 

In Steps 18 and 19, the sum A of the product of the weight computed in S17 and a pixel value of a corresponding 
pixel is computed. The sum B of the weight is computed. Then, A/B is computed to obtain a smoothing value s. 

In Step S20, as in the equation (1 5), a target pixel value and a smoothing value is mixed according to a mixing ratio 
X, to obtain a pixel value of the filtered target pixel. 
so Figure 19 is a flow chart illustrating the outline of the processes shown in Figure 13 through Figure 18. Therefore, 
the corresponding relation alone is illustrated. 
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[Embodiment 2] 



In the embodiment 2, a constitution shown by the solid line 17 in Figure 1 is replaced with that shown by a broken 
line 28. A filter having edge preserving property that is employed as the postfilter 26 is the same as described in the 
25 embodiment 1 This constitution allows to suppress a random noise that occurs in encoding and decoding of a picture, 
and also allows to smooth a picture signal and preserve edge. As a result, predictive error between the present picture 
signal and an input picture signal of the next frame can be decreased, and transmission bit quantity in encoding and 
decoding can be reduced. 

A description will be given of a device and method that performs a processing in the case where a step computed 
30 by the subtracter 1 1 in Figure 1 , is below a certain threshold value, that is, the case where a step is not encoded in the 
encoder 12 in Figure 1 (e.g., a meaningless block in the H. 261 standard). Therefore, this case corresponds to a picture 
of a corresponding picture block which has just being read from the frame memory 18 in Figure 1 . In this case, the pic- 
ture stored in the frame memory has already filtered by the previous frame. Thus, against the picture data of the frame 
memory itself, another filtering is unnecessary. Accordingly, a filtering having edge preserving property is performed 
35 only on a meaningful block whose step has been encoded, and therefore, another filtering is not performed on the iden- 
tical picture, thereby preventing a picture blur. 

A description will be given of the case where the smoothing method of the present invention is applied to an in-loop 
filter (i.e., reference numeral 23 in Figure 1) in the encoding part and the decoding part. 

The numeral 20 in Figure 1 designates a picture signal given motion compensation. Therefore, a quantizing param- 
40 eter and a block activity which are stored by block do not agree with a DCT block division. For this, the value k in the 
equation (16) is fixed, and against the picture block given motion compensation, the equation (14) is computed from the 
equation (1 2). At this time, the motion compensation is computed using a pixel value that the frame memory stores prior 
to the filtering, and an input signal. 



k = k 



fix (33) 



When the block activity is smaller than a certain threshold value 8 2 , provided that: 
when 

b 2 = 8 2' max* 01=8 4 ' (34) 

the mean square error (the equation (35)) of between the input picture and the frame memory is greater than the mean 
square error (the equation (36)) of the input picture and the frame memory after filtering, an in-loop filter in a decoding 
loop is constituted by the equations (15), (18) and (21). 

1 +7 +7 

MSE1 = 64 Z Z W x+1 • v +k)-i(x+1 , y+k)) 2 (35) 
l=o k=0 
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MSE2 = gJXE < m ( x+1 • y+k)-i(x+1 , y + k)) 2 (36) 

l=ok=0 

5 wherein d(x, y) is a picture value that is stored in the frame memory and given motion compensation; i(x, y) is an input 
picture signal; and m(x, y) is a picture value obtained by applying the filter of the present invention to a picture block 
given motion compensation. 

Accordingly, the feature that the in-loop filter in the encoding loop and the decoding loop is constituted by the filter 
having edge preserving property, allows to reduce predictive error in input picture signal between the present frame and 
10 the next frame, thereby decreasing the transmission bit quantity. 

An example of the equations (33) and (34) will be given below. 

X fjx = 0.5, 6 4 =8 (37) 

As described above, according to the embodiment 2, the smoothing filter greatly decreases distortion occurred on 
a block boundary, leading to much visual improvement of picture quality. Further, since the smoothing process is per- 
formed by changing the smoothing filter characteristics according to the roughness of a decoding parameter and the 
activity of a picture signal, it is possible to decrease the resolution of the entire decoded picture, leading to a decoded 
picture that has high quality and is visually legible. Furthermore, since the smoothing filter is used in a decoding loop, 
predictive difference signal becomes small, thereby reducing the transmission bit quantity. 

Claims 

1 . A picture signal smoothing device that performs a smoothing process by applying a smoothing filter that performs 
25 a filtering of a sampled and digitized picture signal by weighted mean of a target pixel and its surrounding pixel, 

including: 

means for detecting a difference between the target pixel and the surrounding pixel; 
means for taking an absolute value of the difference; 

means for adding a non-negative value (m) to the absolute value of the difference thus detected; 
means for taking an inverse number (weighting factor) of the value after the adding; 
means for multiplying a value after taking of the inverse number, and the surrounding pixel value together; 
means for taking a sum (A) of a value after the multiplying; 
means for taking a sum (B) of the inverse number (the weighting factor) alone; and 
means for dividing the sum (A) by the sum (B) of the weighting factor and then normalizing (A/B). 

2. A picture signal smoothing device according to claim 1 including: 

means for diving a picture to be filtered into blocks; and 
40 means for appropriately changing a non-negative value to be added to the difference value between the target 

pixel and the surrounding pixel, according to roughness of quantizing parameter of each block, which is repre- 
sented by quantizing characteristics utilized in encoding or decoding each block of a picture signal. 

3. A picture signal smoothing device according to claim 1 including: 
means for dividing a picture to be filtered into blocks; 

means for computing block activity indicating a change degree of a pixel in each block; and 
means for appropriately changing a non-negative value to be added to the difference value between the target 
pixel and the surrounding pixel, according to the block activity. 

4. A picture signal smoothing device according to claim 1 including: 

means for dividing a picture to be filtered into blocks; 
means for detecting whether a target pixel exists on a block boundary; 
55 means that if the target pixel exists on a block boundary, detects a block activity of a block adjacent to the block 

boundary, i.e., adjacent block activity; and 

means for appropriately changing a non-negative value to be added to the difference value between the target 
pixel and the surrounding pixel, according to the adjacent block activity. 
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5. A picture signal smoothing device according to claim 1 including: 

means for dividing a picture to be filtered into blocks; 

means for mixing a signal value after the smoothing process by block and a signal value prior to the smoothing 
process; and 

means for appropriately changing the mixing ratio according to a roughness of quantizing parameter of each 
block, which is represented by quantizing characteristics utilized in encoding and decoding each block of a pic- 
ture signal. 

6. A picture signal smoothing device according to claim 1 including: 

means for dividing a picture to be filtered into blocks; 

means for mixing a signal value after the smoothing process by block and a signal value prior to the smoothing 
process; and 

means for appropriately changing the mixing ratio according to each block activity in a picture signal. 

7. A picture signal smoothing device according to claim 1 including: 

means for dividing a picture to be filtered into blocks; 

means for mixing a signal value after the smoothing process by block and a signal value prior to the smoothing 
process; 

means for detecting whether a target pixel exists on a block boundary; and 

means that if the target pixel exists on a block boundary, appropriately changes the mixing ratio according to 
an adjacent block activity of a block adjacent to the target pixel. 

8. A picture signal smoothing device that performs a filtering of a sampled and digitized picture signal by a smoothing 
filter, including: 

means for dividing a picture to be filtered into blocks; 

means for detecting whether a target pixel exists on a block boundary; 

means for performing a filtering whose edge preserving property is weakened, if the target pixel exists on a 
block boundary; and 

means for performing a filtering having edge preserving property if the target pixel exists inside a block. 

9. A picture signal smoothing device according to claim 8 including: 

means for dividing a picture to be filtered into blocks; 

means for detecting whether a target pixel exists on a block boundary; 

means for computing an activity of the target pixel from the target pixel value and the surrounding pixel value; 
and 

means that if the target exists on a block boundary, appropriately changes strength of edge preserving prop- 
erty ol a filter according to the activity of the target pixel. 

10. A picture signal smoothing device according to claim 8 including: 

means for dividing a picture to be filtered into blocks; 

means for detecting whether a target pixel exists on a block boundary; 

means for detecting whether a block boundary at which the target pixel exists, continues either of horizontally 
and vertically; and 

means for appropriately changing strength of edge preserving property according to a direction of a block 
boundary to which the target pixel belongs. 

11. A picture signal smoothing method in which a smoothing process is performed by applying a smoothing filter that 
performs a filtering of a sampled and digitized picture signal by weighted mean of a target pixel and its surrounding 
pixel, including the steps of: 

detecting a difference between a target pixel and its surrounding pixel; 
taking an absolute value of the difference; 

adding a non-negative value (m) to the absolute value of the difference thus detected; 
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taking an inverse number (weighting factor) of a value after the adding; 

multiplying a value after taking of the inverse number and the surrounding pixel value together; 
taking a sum (A) of a value after the multiplying; 
taking a sum (B) of the inverse number (the weighting factor) alone; and 
dividing the sum (A) by the sum (B) of the weighting factor and then normalizing (A/B). 

12. A picture signal smoothing method according to claim 1 1 , including the steps of: 

diving a picture to be filtered into blocks; and 
w appropriately changing a non-negative value to be added to the difference value between the target pixel and 

the surrounding pixel, according to roughness of quantizing parameter of each block, which is represented by 
quantizing characteristics utilized in encoding and decoding each block of a picture signal. 

13. A picture signal smoothing method according to claim 1 1 including the steps of: 
dividing a picture to be filtered into blocks; 

computing a block activity indicating a change degree of a pixel in each block; and 

appropriately changing a non-negative value to be added to the difference value between the target pixel and 
the surrounding pixel, according to the block activity. 

14. A picture signal smoothing method according to claim 1 1 including the steps of: 

dividing a picture to be filtered into blocks; 
detecting whether a target pixel exists on a block boundary; 
25 if the target pixel exists on a block boundary, appropriately changing a block activity of a block adjacent to the 

block boundary, i.e.. adjacent block activity; and 

appropriately changing a non-negative value to be added to the difference value between the target pixel and 
the surrounding pixel, according to the adjacent block activity. 

30 15. A picture signal smoothing method according to claim 1 1 including the steps of: 

dividing a picture to be filtered into blocks; 

mixing a signal value after the smoothing process by block and a signal value prior to the smoothing process; 
and 

35 appropriately changing the mixing ratio according to a roughness of quantizing parameter of each block, which 

is represented by quantizing characteristics utilized in encoding and decoding each block of a picture signal. 

16. A picture signal smoothing method according to claim 1 1 including the steps of: 

40 dividing a picture to be filtered into blocks; 

mixing a signal value after the smoothing process by block and a signal value prior to the smoothing process; 
and 

appropriately changing the mixing ratio according to each block activity in picture signal. 
45 17. A picture signal smoothing method according to claim 1 1 including the steps of: 
dividing a picture to be filtered into blocks; 

mixing a signal value after the smoothing process by block and a signal value prior to the smoothing process; 
detecting whether a target pixel exists on a block boundary; and 
so if the target pixel exists on a block boundary, appropriately changing the mixing ratio according to an adjacent 

block activity of a block adjacent to the target pixel. 

18. A picture signal smoothing method in which a filtering of a sampled and digitized picture signal is performed by a 
smoothing filter, including the steps of: 

55 

dividing a picture to be filtered into blocks; 

detecting whether a target pixel exists on a block boundary; 

performing a filtering whose edge preserving property is weakened, if the target pixel exists on a block bound- 
ary; and 
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performing a filtering having edge preserving property if the target pixel exists inside a block. 

19. A picture signal smoothing method according to claim 18 including the steps of: 

s dividing a picture to be filtered into blocks; 

detecting whether a target pixel exists on a block boundary; 

computing an activity of the target pixel from a target pixel value and its surrounding pixel value; and 
appropriately changing strength of edge preserving property of a filter according to the activity of the target 
pixel. 

10 

20. A picture signal smoothing method according to claim 18 including the steps of: 

dividing a picture to be filtered into blocks; 
detecting whether a target pixel exists on a block boundary; 

detecting whether a block boundary at which the target pixel exists, continues either of horizontally and verti- 
cally; and 

appropriately changing strength of edge preserving property according to a direction of a block boundary to 
which the target pixel belongs. 

21 . A picture encoding device in which a filter is attached to a decoding loop having a frame memory for storing a signal 
that corresponds to either of at least frame and at least one field, and an inter-frame encoding between is per- 
formed by a corresponding pixel between an input signal and the frame memory, is characterized in that: 

a filter having edge preserving property is employed as the above filter; and 

a picture signal after being processed in a picture signal smoothing device is input to the frame memory storing 
a picture signal of a preceding frame, as a local decoding signal. 
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22. A picture encoding device according to claim 21 including: 

means for dividing a picture into blocks and computing an inter-frame difference by block; 
means for determining whether the difference exceeds a certain threshold value; and 
means that only when the difference exceeds the threshold value, a corresponding block is processed by a fil- 
ter having edge preserving property. 

23. A picture encoding device in which a filter is attached to a decoding loop having a frame memory for storing a signal 
that corresponds to either of at least one frame and at least one field, and an inter-frame encoding is performed by 
a corresponding pixel between an input signal and the frame memory, including: 

means for computing a motion vector; 
40 means for reading a picture value stored in the frame memory with consideration of the motion vector; and 

a filter having edge preserving property, which filter processes an output signal of the frame memory. 

24. A picture decoding device in which a filter is attached to a decoding loop having a frame memory for storing a signal 
that corresponds to either of at least one frame and at least one field, and an inter-frame decoding is performed by 

45 a corresponding pixel between an input signal and the frame memory, is characterized in that: 

a filter having edge preserving property is employed as the above filter; and 

a picture signal after being processed in a picture signal smoothing device is input to a frame memory storing 
a picture signal of the previous frame, as a decoding signal. 



50 



25. A picture decoding device according to claim 24 including: 



means for dividing a picture into blocks and computing an inter-frame difference by block; 
means for determining whether the difference exceeds a certain threshold value; and 
55 means that only when the difference exceeds the threshold value, a corresponding block is processed by a fil- 

ter having edge preserving property. 

26. A picture decoding device in which a filter is attached to a decoding loop having a frame memory for storing a signal 
that corresponds to either of at least one frame and at least one field, and an inter-frame decoding is performed by 
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a corresponding pixel between an input signal and the frame memory, including: 
means for computing a motion vector; 

means for reading a picture value stored in the frame memory with consideration of the motion vector; and 
5 a filter having edge preserving property, which filter processes an output signal of the frame memory. 

27. A picture encoding method in which a filter is attached to a decoding loop having a frame memory for storing a sig- 
nal that corresponds to either of at least one frame and at least one field, and an inter-frame encoding is performed 
by a corresponding pixel between an input signal and the frame memory, including the steps of: 

10 

performing a process using a filter having edge preserving property; and 

inputting a picture signal after being processed in a picture signal smoothing device to the frame memory stor- 
ing a picture signal of a preceding frame, as a local decoding signal. 

is 28. A picture encoding method according to claim 27 including: 

dividing a picture into blocks and computing an inter-frame difference by block; 
determining whether the difference exceeds a certain threshold value; and 

only when the difference exceeds the threshold value, a corresponding block is processed by a filter having 
20 edge preserving property. 

29. A picture encoding method in which a filter is attached to a decoding loop having a frame memory for storing a sig- 
nal that corresponds to either of at least one frame and at least one field, and an inter-frame encoding is performed 
by a corresponding pixel between an input signal and the frame memory, including the steps of: 

25 

computing a motion vector; 

reading a picture value stored in the frame memory with consideration of the motion vector; and 
processing an output signal of the frame memory by using a filter having edge preserving property. 

30 30. A picture decoding method in which a filter is attached to a decoding loop having a frame memory for storing a sig- 
nal that corresponds to either of at least one frame and at least one field, and an inter-frame decoding is performed 
by a corresponding pixel between an input signal and the frame memory, including the steps of: 

performing a process using a filter having edge preserving property; and 
35 inputting a picture signal after being processed in a picture signal smoothing device to the frame memory stor- 

ing a picture signal of a preceding frame, as a decoding signal. 

31 . A picture decoding method according to claim 30 including the steps of: 

40 dividing a picture into blocks and computing an inter-frame difference by block; 

determining whether the difference exceeds a certain threshold value; and 

only when the difference exceeds the threshold value, a corresponding block is processed by a filter having 
edge preserving property. 

45 32. A picture decoding method in which a filter is attached to a decoding loop having a frame memory for storing a sig- 
nal that corresponds to either of at least one frame and at least one field, and an inter-frame decoding is performed 
by a corresponding pixel between an input signal and the frame memory, including the steps of: 

computing a motion vector; 
so reading a picture value stored in the frame memory with consideration of a motion vector; and 

processing an output signal of the frame memory by using a filter having edge preserving property. 
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